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ABSTRACT 



This report describes options for study designs to evaluate 
the effect of early intervention on infants and toddlers, birth to three, 
with disabilities or developmental delays and their families in Washington 
state. Information was gathered from extensive literature reviews, a survey 
of local assessment efforts in Washington, a technical data inventory, and a 
consultation with stakeholders such as service providers, agency personnel, 
the Office of Superintendent of Public Instruction, parents, and community 
supporters. Recommendations include: (1) using a small- scale longitudinal 

study, retrospective cross-sectional study, or a key informant study for 
early intervention research; and (2) using an integrated research plan that 
logically and methodically addresses: who the children and families are who 
are receiving services, what are their needs, how services are addressing the 
needs of children and families, and what child and family outcomes occur as a 
result of service provision. Appendices include information on child and 
family characteristics, a local assessment survey, state and national early 
intervention goals, and possible statistical models for using subjects as 
their own control. (Contains over 150 references.) (CR) 
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INDICATORS FOR CHILD AND FAMILY OUTCOMES 



CHILD OUTCOMES/INDICATORS 


FAMILY OUTCOMES/INDICATORS 


Developmental Skills 


Knowledge 


* cognitive development 


* knowledge of child’s disability 


* communication 


* understanding what the child needs to 


* social competence (social/emotional) 


enhance development 


* adaptive (level of independence) 

* physical (motor, vision, hearing) 


* ability to read child’s cues 

Empowerment 


Health 


* confidence in ability to enhance 


* children have the best possible health 


child’s development 
* confidence in ability to interact with 


Readiness to Learn 


professionals and advocate for 


* confidence 


services 


* curiosity 

* persistence 


* locus of control 


* capacity to communicate 


Parent (Primary Caregiver)-Child 


* capacity to get along with others 


Interaction 

* reciprocity 


Child-Parent (Primary Caregiver) 
Interaction 


* responsivity 


* reciprocity 


Support 


* responsivity 


* family has economic resources 

* family has social support 

Impact of Early Intervention 

* perceived impact of services on child 

* perceived impact of services on family 

* perception of services delivered 




viii 



EVALUATION OF EARLY INTERVENTION IN 
WASHINGTON STATE: OPTIONS AND ISSUES 

EXECUTIVE SUMMARY 



This report describes options for study designs to evaluate the impact of early intervention 
on infants and toddlers, birth to three, with disabilities or developmental delays and their 
families, one of Washington State’s most vulnerable populations. 

Information was gathered from an extensive literature review, a survey of local assessment 
efforts in Washington State, a technical data inventory, and consultation with stakeholders 
such as service providers, agency personnel (Department of Health, Department of Social 
and Health Services, and the Office of Superintendent of Public Instruction), parents, and 
community supporters. Research and Data Analysis convened an advisory group to 
provide professional expertise in reviewing and commenting on materials prepared for this 
project. 

The project was funded by the Department of Social and Health Services (DSHS), Infant 
Toddler Early Intervention Program (ITEIP). Its intent was to develop study designs to 
meet the evaluation goals of parents, the Joint Legislative Audit and Review Committee 
(JLARC), ITEIP, and stakeholders. 



OUTCOMES 

The study designs, and their associated child and family outcomes, are intended to address 
the question: what is the impact of early intervention services on children, birth to three, 
with disabling or delaying conditions and their families in Washington State? The range 
of outcomes chosen for the child and the family and indicators are listed on the previous 
page. The particular design of the study determines which outcomes are selected for that 
study. 



STUDY DESIGN OPTIONS 

Research on the effectiveness of early intervention involves a variety of methodological 
challenges. The absence of a control group presents the greatest problem. If subjects 
cannot be randomly assigned to treatment (early intervention) and non-treatment (no early 
intervention), it is difficult to conclude that changes in child/family outcomes are due 
specifically to early intervention. To control for effects on outcomes from factors other 
than early intervention, characteristics which affect child/family outcomes must be taken 
into account. 
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The main study designs are as follows: 



STUDY DESIGN TYPES 

® Child/Family Serves as Own Control — Large Scale Longitudinal Study 
® Child/Family Serves as Own Control — Small Scale Longitudinal Study 

• Comparison Group — Longitudinal Study 
© Pre-Post Study 

• Retrospective Cross-sectional Study 

• Key Informant Study — Survey-only and Three-tiered Approach 



Design options include the use of each child/family as its own control, which means there 
is no control or comparison group. These studies essentially measure changes in each 
individual and family that have occurred during the course of intervention, and compare 
beginning outcome data to outcome data at future points in time for each subject. Other 
studies seek to identify appropriate comparison groups. Some design options use 
perceptions of key informants and information provided in state data bases. 

OUTCOME STUDY RECOMMENDATIONS 

• Research and Data Analysis and the Advisory Group were in agreement in 
recommending the design, Child/Family Serves as Own Control — Small Scale 
Longitudinal Study. 

While this study design offers many advantages, its estimated direct cost, at least $1.4 
million, and the availability of results, not before the year 2004, caused Research and 
Data Analysis and the Advisory Group to include lower cost options that would 
provide more timely results. 

If budget and timeliness are constraints: 

• Research and Data Analysis recommends the Retrospective Cross-sectional Study as a 
less expensive and shorter timeline option. 

• The majority of the Advisory Group recommends the Key Informant Study — Three- 
tiered Approach. 



GENERAL RECOMMENDATION 



The need for a more comprehensive alternative approach to evaluation of early 
intervention in Washington became evident during the process used to complete this 
report. On the basis of all the information gathered during this process, Research and 
Data Analysis recommends: 



• An integrated research plan that logically and methodically addresses four 
questions: 



=> Step One: Who are the Children and Families Receiving Services? 

Use enhanced profiles in annual child count to describe the children and 
families who are receiving early intervention in Washington State. 

=> Step Two: What Are Their Needs? 

Conduct a needs assessment of children and families who are receiving 
early intervention services in Washington State using focus groups, 
interviews, and surveys. 

=> Step Three: How Are Services Addressing The Needs of Children and 



=> Step Four: What Child and Family Outcomes Occur as A Result of Service 
Provision? 

Conduct an outcome study which identifies positive child and family 
outcomes occurring as a result of early intervention services in 
Washington State. 

In evaluating early intervention services, decision makers should determine what specific 
information they seek, at what price, and how long they wish to wait for the results. Once 
these parameters are defined, the information presented in this report can be used to make 
a well-informed decision as to how to evaluate early intervention services. 



Families? 



Conduct a review of program services provided to children with 
disabling or delaying conditions and their families in Washington State 
using monthly service reports and program observation. 
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CHAPTER 1 



INTRODUCTION 

This report describes options for study designs to evaluate the impact of early intervention 
on infants and toddlers, birth to three, with disabilities or developmental delays and their 
families, one of Washington State’s most vulnerable populations. 

The Child and Family Early Intervention Outcome Measures Planning Project is funded 
by the Department of Social and Health Services (DSHS), Infant Toddler Early 
Intervention Program (ITEIP). This project’s intent was to develop study designs to meet 
the evaluation goals of parents, the Joint Legislative Audit and Review Committee 
(JLARC), ITEIP, and stakeholders. This report presents a range of study designs with 
outcome measures, data sources and methods needed to accomplish them, and their 
approximate costs. 

It is hoped that the outcome measures and study designs presented here will enhance 
planning and discussion at the state and local levels, and facilitate decision making and 
priority setting for Washington’s early intervention programs for infants and toddlers and 
their families. In addition to the outcome measures and study designs, summaries of our 
preliminary background research are presented in the separate appendix. Literature 
Summary, and in this report. These include the literature summary (separate appendix), 
local assessment (Chapter 5), and data inventory (Chapter 6). 

BACKGROUND 

With the passage of the Education for All Handicapped Children Act (Public Law 94-142) 
in 1975, later retitled the Individuals with Disabilities Education Act (IDEA), the 
educational rights for all school-age children with disabilities, ages 6 through 1 8, were 
ensured. Within this law were incentives for including preschool-age children. This 
legislation evolved from various state statutes and federal court cases based on the United 
States Constitution (Martin et al., 1996). 

Policy makers at the federal level, recognizing the importance of early referral and 
intervention for children with disabilities and developmental delays ages birth to five, 
passed the 1986 amendments to Public Law 99-457 later known as the Individuals with 
Disabilities Education Act (IDEA). Part B of IDEA extends rights to children with 
disabilities, ages three to six.* Part H of IDEA offers states federal incentive funds to plan, 
develop, and implement a statewide system of coordinated interagency programs for 
infants and toddlers with disabling or delaying conditions, birth to three years of age, and 
their families. The Part H program in Washington State is named the Infant Toddler Early 
Intervention Program (ITEIP). The Governor designated DSHS as the lead agency for 
Part H federal funds. The five participating state agencies include DSHS; Department of 
Community, Trade and Economic Development; Department of Health; Department of 



’ In this report, three to six indicates children who are three years or older, and less than six years of age. 
(Office of Superintendent of Public Instruction refers to these children as 3 to 5 year olds.) 

O 

ERIC 



12 



Services for the Blind; and Office of the Superintendent of Public Instruction. In October 
1994, Washington State began full implementation of Part H. As of July 1998, Part H will 
be titled Part C. 



In Washington State, 5,007 infants and toddlers under three years of age were determined 
to be enrolled in public early intervention services for delaying or disabling conditions as 
of December 1, 1997 (Lyons et al., 1998). Infants and toddlers with disabilities and their 
families are eligible to receive an array of public early intervention services although all 
may not seek enrollment in state programs. Public early intervention services for children 
birth to three with delaying or disabling conditions are provided by child development 
centers, county health and human service agencies, educational service districts, 
neurodevelOpmental centers, school districts, tribal programs, and other local and state 
agencies. The Individuals with Disabilities Education Act (IDEA), Part H program acts as 
an umbrella and provides linkages and enhancement of these services to ensure a statewide 
system of comprehensive, multi-disciplinary, coordinated services to infants and toddlers 
with disabilities and their families. 

In the Birth to Three Years Study (Harding and Keating, 1995), the Legislative Budget 
Committee (now called the Joint Legislative Audit and Review Committee, JLARC) and 
the Washington State Institute for Public Policy (WSIPP) recommended that the 
legislature conduct an evaluation of the Part H program and that the Department of Social 
and Health Services and the State Interagency Coordinating Council develop outcome 
measures and begin tracking the necessary data to document these measures. 

Measurement of Impact of Part H and Early Intervention Services 

In the early stages of this project many long discussions including those with the Advisory 
Group revolved around whether the most appropriate response to JLARC’s 
recommendation was to measure the impact of Part H or the public early intervention 
services. It was decided that these questions could not be separated, in that Part H is an 
integral part of early intervention services in Washington State. 

Measures of Success 



Neurophysiological Basis of Future Development 



ERIC 



A new consensus about the importance of intervening in the first months and years of life 
is emerging in the field of early intervention for children with established disabilities or 
developmental delays and those at risk biologically or environmentally (Guralnick, 1998). 
In 1997 House Resolution 131 was submitted entitled Zero to Three Resolution for Early 
Childhood Development, expressing the consensus of the House of Representatives that 
sufficient handing should be provided to meet the needs of infants and toddlers through 
five programs including Part H of Individuals with Disabilities Education Act (IDEA). 
This resolution acknowledged the critical nature of the first three years of life: 
“groundbreaking scientific research shows that children’s experiences during their first 
three years of life determine how the intricate circuitry of their brains are ‘wired’ and lay 
the foundation for their future development.” 
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This statement is indeed borne out in the research literature which shows that during 
infancy and early childhood the connections between neurons in the brain, or neural 
synapses, reach their highest density, well above that of adults, and remain at that level 
until late childhood (Chugani et al., 1987; Huttenlocher, 1990). This is a period of great 
potential, for it is sensory experience that helps determine the pattern of wiring between 
the neurons in the brain (Greenough and Black, 1992; Weiler et al., 1995). This link 
between brain activity and brain structure points to the importance of the critical early 
years of life. 

Child Development and Parent-Child Relationship 

What the brain learns determines the individual child’s responses to the environment. The 
sensory experience that embodies success to an infant or toddler and lays the foundation 
for future learning may be the gentle and kind touch of her mother, the extended gaze of 
his father, the soft, soothing voice of his caregiver, and never being taught to feel less than 
her full potential. Research studies report the long-term significance of the early parent- 
child relationship and its power to facilitate a child’s successful development (Ainsworth, 
1973;Bowlby, 1969; Lyons-Ruth and Zeanah, 1993; Tjossem, 1976). Feeling loved and 
valued and knowing that their needs matter and will be met foster the development of 
trust, self-confidence, curiosity, and empathy (Carnegie Task Force on Meeting the Needs 
of Young Children, 1994). With this foundation, these infants and toddlers are then free 
to explore and discover. Understanding a child’s responses and development is necessary 
for families to actively support the child in there endeavors. 

PartH 

This concept of the critical importance of parents and other caregivers is reflected in the 
vision and goals of Individuals with Disabilities Education Act (IDEA), Part H. The 
delivery of family-centered early intervention services required by Part H relates to the 
new direction in early intervention which focuses on enhancing the quality of early parent- 
infant interactions (Bennett and Guralnick, 1991). One of the general state program goals 
per the national legislation for IDEA, Part H (1997) is “a family-directed assessment of the 
resources, priorities, and concerns of the family and the identification of the supports and 
services necessary to enhance the family’s capacity to meet the developmental needs of the 
infant or toddler.” 

The general goals of Washington’s Infant Toddler Early Intervention Program (Part H) 
include the coordination of development and implementation of interagency and 
multidisciplinary delivery of early intervention services which enhance the development of 
each infant and toddler with disabilities; provide services that the child’s family defines 
according to the family’s concerns, priorities, and resources; and provide family support 
for enhancing the child’s development. 



Family Measures of Success 



Underlying the focus on a family-centered approach is the established fact that the quality 
of the early parent-child relationship is related to child development and will affect child 
outcomes whether they are measured at three years of age or at six years of age. For 
many researchers the focus has shifted to determining what the family outcomes in early 
intervention would be from this family-centered approach. Bailey and his colleagues 
(1998) are among the principal investigators for the National Early Intervention 
Longitudinal Study (NEILS) involving twenty states. Their recent publication suggests 
eight questions as a framework for the assessment of family outcomes from early 
intervention. These questions focus on family perceptions of the early intervention 
experience and on the impact on the family. Questions designed to assess the impact on 
the family include focus on the extent to which early intervention fosters parents’ : 

1) perceived competence as caregivers; 2) ability to work with professionals and advocate 
for services (empowerment); 3) informal social and community support systems; 4) quality 
of life; and 5) optimism for a positive future, which Turnbull and Turnbull (1997) point 
out requires early interventionists’ promotion of an optimistic perspective by 
acknowledging positive possibilities for the toddler and encouraging “great expectations.” 

Measures of success that emerged from input we received from individual parents and 
family members of children with delaying or disabling conditions were quality of parent- 
child interaction, information about child’s disability, a sense of hope, adequate 
information to make decisions, inclusion of parent’s perspectives of the outcomes on the 
Individualized Family Service Plan (IFSP), and knowledge of specific techniques to 
enhance their child’s development at home. 

Child Measures of Success 

Traditionally, changes in cognitive development, measured by standardized tests, were 
believed to be the most important measure of success. A general decline in the intellectual 
development of children with established disabilities and those at risk occurs in the 
absence of early intervention. Unequivocal evidence now exists that this decline can be 
substantially reduced by interventions implemented during the first 5 years of life 
(Guralnick, 1998). 

Outcome measures from the primary domains of cognitive, language, affective, and motor 
development should be expanded to include measures that reflect a broader vision of the 
goals of early intervention such as social competence and improvement of children’s health 
status (Guralnick and Neville, 1997). Peck et al. (1993) and Billingsley et al. (1996) 
suggest a more global measure — participation in valued roles, activities, and 
settings — which includes the related and more specific outcomes of memberships, 
relationships, and skills. 
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Children’s capacity for development is facilitated in the context of secure attachments; 
consistent, warm nurturing relationships with caregivers; stimulating environments, 
including a variety of developmental^ appropriate toys and materials; quality social 
contact with other adults and children; a general state of health; and a safe environment 
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(Guralnick, 1998). Parents and other adult caregivers play critical roles in the child’s 
development during these early years (Carnegie Task Force on Meeting the Needs of 
Young Children, 1994; Shonkoff et al., 1992). Measuring success requires a framework 
which includes both child and family outcomes and considers the factors in the conceptual 
model presented on the following two pages. 
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CONCEPTUAL MODEL 



Early Intervention Services 
Components 
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FOR IMPACTS OF EARLY INTERVENTION 






Participation in valued 
roles, activities, 
and settings 





CHAPTER 2 



METHODS 



The study designs and outcome measures presented in this report were based on 
preliminary background research, input from key informants, and the combined years of 
research experience of the project team. 

PRELIMINARY BACKGROUND RESEARCH 

Preliminary Background Research included the literature review, local assessment and 
data inventory. 

Literature Review 

A literature review of evaluation methods and measures in the child development/early 
intervention field was performed. This review’s intent was to help identify the questions 
which are most critical, the evaluation methods and measures which have been most 
successful in assessing the impact of early intervention programs, the currently preferred 
methodologies in the child development/early intervention field, and major difficulties with 
alternative evaluation strategies. Over 250 published papers on evaluation of early 
intervention/child development were reviewed. 

The bibliography and literature summary table resulting from this literature review can be 
found in a separate publication. Evaluation of Early Intervention in Washington State 
Literature Summary. The bibliography notes a wide variety of articles useful in 
understanding issues of evaluation methodologies and outcomes for children with 
disabilities and their families. The literature summary table includes some key sources of 
insight into current evaluation and outcome recommendations and strategies, and 
examples of recent studies and those currently underway. Lists of child, family, and 
program characteristics from this literature review were made and are included in 
Appendix B. 

The outcome, parent-child interaction, comes from a specific body of research literature. 

In this report the use of the word parent refers to the primary caregiver if other than the 
parent. 

Local Assessment 

The Local Assessment Survey (Appendix C) was developed to provide knowledge of 
procedures being used by communities and programs in Washington State for assessing 
local early intervention efforts. The survey was mailed to child development centers, 
County Interagency Coordinating Council (CICC) chairs, Infant Toddler Early 
Intervention Program (ITEIP) contractors, neurodevelopmental centers, school districts, 
tribal authorities, and universities requesting information on assessment, tracking, or 
research efforts being undertaken. Chapter 5 describes this process in greater detail. 
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Technical Data Inventory 



A technical inventory was done to identify available statewide data which could be linked 
at the individual level to study the impact of early intervention programs. Assessment of 
the feasibility of collecting this information was investigated. 

INPUT FROM STAKEHOLDERS 

Advisory Group 

The Research and Data Analysis project team convened an Advisory Group to provide 
professional expertise in reviewing and commenting on materials prepared for this project, 
developing study designs, and assessing their feasibility. Membership in the Advisory 
Group included expertise in the areas of genetics, nursing, pediatrics, preventive medicine, 
research, and Special Education (please see Appendix A for biosketches on individual 
Advisory Group members). 

Parents 

Unstructured interviews were conducted with individual parents of children with disabling 
or delaying conditions, including Cassie Johnston, Parent Participation Coordinator for 
Washington State. Participation in one of the four State Interagency Coordinating 
Council Conversations with Families and summative information from the other three 
provided another source of data. 

Additional input was received from one of our Advisory Group members who is both a 
parent of a child who was enrolled in early intervention services and a Family Resource 
Coordinator. 

Service Providers 

One of the data sources for the development of outcome measures and study designs was 
a site visit to the Tacoma Learning Center. Research and Data Analysis staff met with 
Maryanne Barnes, M.Ed., executive director. 

Input was additionally received from a member of the Advisory Group, Judith Moore, 
M.A., executive director of Boyer Children’s Clinic and former Chair of the State 
Interagency Coordinating Council. 

Agency Personnel 

Representative input from public agencies in Washington State included members of the 
Advisory Group from the Department of Health (DOH), Department of Social and Health 
Services (DSHS), and the Office of Superintendent of Public Instruction (OSPI). 

Background research concerning available data in Washington State included informant 
interviews with potential data sources such as the Department of Health and the Office of 
Superintendent of Public Instruction. 
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THE STATE INTERAGENCY COORDINATING COUNCIL (SICC) 



The State Interagency Coordinating Council (SICC) for Infants and Toddlers with 
Disabilities and Their Families was involved in the development of this project beginning 
with the presentation of the proposal for the Child and Family Early Intervention 
Outcome Measures Planning Project in July 1997, received a status report in April 1998, 
and reviewed the draft for this report in June 1998. 
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CHAPTER 3 



DEVELOPMENT OF FAMILY AND CHILD OUTCOME 
MEASURES AT THE FEDERAL LEVEL 



INTRODUCTION 

This section examines steps taken by the Federal Government to develop early 
intervention outcome measures for children with disabling and delaying conditions and 
their families. The Federal Government funds both national studies, which focus on 
specific outcomes developed by national committees, and state and individual research 
projects which focus on specific topics in early intervention. 

The Federal Government monitors state implementation of early intervention programs. 

As a condition of state participation in Part H, the lead agency in each state is required to 
submit annual state data and performance reports which include information on the 
following: the number of children served under Part H; the number of infants under one 
year served under Part H; the settings in which services are provided; funding sources; and 
state monitoring activities [US. Department of Education Program Performance Plans 
(FY1999), 1998], The Federal Government also has funded implementation research such 
as the Early Childhood Research Institute’s Service Utilization project which described the 
services provided to infants, toddlers, and preschool children with disabilities and their 
families across nine communities in three states (Colorado, Pennsylvania, and North 
Carolina) (Gloria Harbin, Thomas Kochanek, cover letter to report, September 1997, 
Frank Porter Graham Child Development Center). 

At this time, the Federal Government does not require participating Part H states to 
include child and family outcomes in their annual reports. The Government is, however, 
developing child and family outcome measures to include as part of a list of Part H 
indicators through its national longitudinal study. National Early Intervention Longitudinal 
Study (NEILS) funded by the Office of Special Education Programs, U S. Department of 
Education. It can also draw from results of other federally funded projects to determine 
which child and family outcomes might be appropriate for monitoring long term. 

The following federally funded projects include family and child outcomes as an integral 
part of their studies. 

NATIONAL EARLY INTERVENTION LONGITUDINAL STUDY (NEILS) 

Status: In progress 



General Description 

The National Early Intervention Longitudinal Study (NEELS) is a five year longitudinal 
study designed to provide both descriptive and explanatory information regarding process 
outcomes and family and child outcomes using the following main study questions: 
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® Who are the children and families receiving early intervention services? 

• What early intervention services do participating children and families receive, and 
how are those services delivered? 

® What outcomes do participating children and families experience? 

• How do outcomes relate to variations in child and family characteristics and services 
provided? 

Data will be collected from the following sources: 

NEILS: SOURCES AND TYPES OF DATA INFORMATION 



DATA SOURCE 


DATA TYPES 


Telephone interviews with families 


child and family characteristics 

child functioning 

families’ perception of services 


Semi-annual reports from service providers 


services provided to children and families 
information on children’s transitions 


Survey of service providers 

\ 


their background 
their training 

ways they deliver services 


Profiles of local state early intervention 
system elements 


elements of delivery system 
costs 


Survey of teachers 


progress of NEILS children at age 5 
services being provided at age 5 


Survey of early intervention program 
directors 


program description 



The study will include a nationally representative sample involving three to five counties in 
twenty states. Approximately 3,800 children will be included. Children between birth and 
3 1 months who are newly entering early intervention will be recruited (1997-1998). The 
project is being conducted by a national investigative team (Stanford Research Institute 
International, Frank Porter Graham Child Development Center, Research Triangle 
Institute, American Institute for Research) and a national advisory panel. 
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NEILS PROJECT STAFF AND PARTICIPATING STATES 



NEILS National 
Advisory Panel 


NEILS Investigators 


NEILS Participating 
States 


James Blackman 


SRI International 


Arkansas 


Susan Bubb 


Kathleen Hebbeler 


California 


Carl Dunst 


Patricia McKenna 


Colorado j 


Mary Elder 


Donna Spiker 


Florida 


Linda Goetze 


Mary Wagner 


Hawaii 


Gloria Harbin 




Idaho 


Kofi Marfo 


Frank Porter Graham Child 


Illinois 


Samuel Meisels 


Development Center 


Kansas 


Luzanne Pierce 


Don Bailey 


Maryland 


Aida Ramirez 


Lynne Kahn 


Massachusetts 


Cordelia Robinson 


Robin McWilliam 


Michigan 


Arnold Sameroff 


Rune Simeonsson 


Minnesota 


Nancy Sweet 




New Mexico 




Research Triangle Institute 


New York 




Susan Kinsey 


North Carolina 
Ohio 




American Institutes for Research 


Pennsylvania 




Jay Chambers 


South Carolina 




Thomas Parris 


South Dakota 
Texas 



(NEILS web site: http//www.sri.com/policy/cehs/neils/neilprog.html, 3/13/98; NEILS 
Draft Surveys, February 27, 1997) 



Child and Family Outcomes 

While specific child and family outcomes and specific analytic methods have not yet been 
finalized, general outcome measures and data sources have been suggested. 

Family outcomes emerged as a result of the work conducted by NEELS investigators. The 
process involved a literature review; discussions with colleagues, practitioners, and family 
members; and discussions among NEELS investigators. The results of these efforts 
produced two outcome domains and eight evaluative questions (Bailey et al., 1998). 
Outcomes data will be gathered using the data sources already mentioned. 
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NEILS FAMILY OUTCOMES 



| FAMILY OUTCOMES 


QUESTIONS 


Family Perceptions of the Early 
Intervention Experience 


1 . Does the family see early intervention as 
appropriate in making a difference in their child’s 
life? 

2. Does the family see early intervention as 
appropriate in making a difference in their 
family’s life? 

3 . Does the family have a positive view of 
professionals and the special service system? 


Impact on the Family 


1 . Did early intervention enable the family to help 
their child grow, learn and develop? 

2. Did early intervention enhance the family’s 
perceived ability to work with professionals and 
advocate for services? 

3. Did early intervention assist the family in building 
a strong support system? 

4. Did early intervention help enhance an optimistic 
view of the future? 

5. Did early intervention enhance the family’s 
perceived quality of life? 



Child outcomes will involve the domains of child functioning, quality of life, and service 
needs (Personal E-mail Communication, Kathy Hebbeler, NEELS coordinator, 4/17/98). 

NEILS CHILD OUTCOMES 



CHILD OUTCOMES 


DATA SOURCE 


Child Functioning Within Specific Developmental Domains 


Primary 


Quality of Life 


Family telephone interview 


Child Engagement (how child interacts with environment) 


Secondary 


Progress toward IFSP goals 


Service provider report/survey 


Service Needs At Ages 3 and 5 





It should be noted that the selection of child and family outcomes has not been finalized. 
It is estimated that NEILS will be completed in 2003. 
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EARLY CHILDHOOD RESEARCH INSTITUTE ON MEASURING GROWTH 
AND DEVELOPMENT (ECRI-MGD) 

Status: In Progress 

General Description 

This study is funded by Early Education Programs for Children with Disabilities at the 
Office of Special Education and Rehabilitative Services (OSERS), U.S. Department of 
Education. The goal of the measurement and growth project is to produce a 
comprehensive, individualized measurement system for children with disabilities from birth 
to eight years of age. The system will include the following: 1) indicators for monitoring 
progress and 2) solutions-oriented assessments to allow family and educators the 
opportunity to optimize child development outcomes. 

Investigators from the Universities of Minnesota, Kansas, and Oregon are participating in 
the study. Five major products of the study include the following: 1) selection of general 
outcomes for children birth to age eight; 2) identification and evaluation of individual 
growth and development indicators; 3) development of exploring solutions assessments; 

4) field testing and evaluations of “consumer satisfaction;” and 5) dissemination 
(http://www.ici. coled. umn.edu8001/ecri, 2/2/98). To date, the institute has produced a 
draft report on the selection of general growth outcomes discussed below. 



ECRI PROJECT STAFF 



ECRI-MGD Investigators 


ECRI-MGD Research Staflf/Assistants 


Center for Early Education and Development and 
Institute on Community Integration 
University of Minnesota 
Scott McConnell 
Mary McEvoy 


With assistance from: 

University of Minnesota 
James Ysseldyke 

Parent Advocacy Coalition for 
Educational Rights (PACER) 
Paula Goldberg 


Juniper Gardens Children's Project, University of 
Kansas 


Research Staff 

University of Kansas 


Judith J. Carta 


Jane Atwater, Dale Walker . 


Charles R. Greenwood 


Carol Leitschuh, Gayle Luze 


Department of Applied Behavioral and 


University of Minnesota 
Jeff Priest , Kristen Davis 


Communication Sciences 


Stephanie McNeill 


University of Oregon 
Ruth Kaminski 


University of Oregon 


Roland H. Good 111 


Deborah Laimon 


Mark Shinn 









17 



26 



Child and Family Outcomes 

The focus of ECRI’s first technical report is the selection of outcomes for children with 
disabilities from birth to eight. Family outcomes are beyond the purview of this technical 
report. Investigators approached the selection of child outcomes by seeking to validate 
outcomes after a “comprehensive review and synthesis of existing outcome statements” 
(Early Childhood Research Institute on Measuring Growth and Development, Draft 
Report, 1998). While the aim of the research team was to produce general outcomes for 
the age group of birth to eight years, they divided this age group into three subgroups: 
birth to two, three to five, and five to eight years. After developing specific outcome lists 
for each age group, these lists were then cross-referenced to produce a list of common 
outcomes for children aged birth to eight. 

The research team formulated outcomes within the parameters of the social context within 
which they would be evaluated. These parameters included two main elements: 1) “the 
impact of outcomes on understanding and enhancing the development trajectories of 
young children with and without disabilities;” and 2) “how early childhood constituencies 
would perceive the appropriateness and usefulness of these outcomes” (Early Childhood 
Research Institute on Measuring Growth and Development, Draft Report, 1998). The 
investigators especially kept in mind the fact that “the future use of the general growth 
outcomes would hinge greatly on acceptance by families, teachers, administrators, and 
other individuals involved in early childhood education” (Early Childhood Research 
Institute on Measuring Growth and Development, Draft Report, 1998). The research 
team plans to submit the outcomes to early childhood constituencies for review. 

(see ECRI Outcomes Table, Next Page) 

The research team is in the process of testing various indicators and measuring processes 
for the above outcomes using a series of formative pilot studies employing different data 
collection formats such as naming pictures, playing with a peer, telling a story, as well as 
different scoring techniques. Investigators “expect to continue evaluating promising 
indicators empirically over the next two years, at least” (Personal E-mail Communication, 
Jeff Priest, ECRI research staff, 4/3/98). 

The general aim of the project is to produce a set of outcomes for children and families 
with delaying or disabling conditions that can be monitored over time (specifically birth to 
age eight) and measured reliably. If the project is successful, it will address two of the 
most pressing problems in early childhood education research — measurement reliability 
and comprehensive outcomes conducive to longitudinal monitoring. 
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ECRI: CHILD OUTCOMES 



OUTCOMES . 

The child uses language to convey and comprehend communicative and social intent 

Child uses gestures, sounds, words, or sentences to initiate, respond to, or maintain word 
combinations. 

Child uses gestures, sounds, words, or sentences to initiate, respond to, or maintain reciprocal 
interactions with others. 



The child takes responsibility for their behavior, health, and well-being, even in the face of 
challenge or adversity 

Child engages in a range of basic self-help skills, including but not limited to skills in dressing, 
eating, toileting/hygiene and safety/identification. 

Child meets behavioral expectations (such as following directions, rules, and routines) in home, 
school, and community settings. 

Child appropriately varies or continues behavior to achieve desired goals. 



The child negotiates and manipulates the environment 

Child moves in a fluent and coordinated mannet to play and participate in home, school, and 
community settings. 

Child manipulates toys, materials, and objects in a fluent and coordinated manner to play and 
participate in home, school, and community settings. 



The child initiates, responds to, and maintains positive social relationships 

Child interacts with peers and adults, maintaining social interactions and participating socially 
in home, school, and community settings. 

Child appropriately solves problems in his/her interactions with others. 

Child shows affect appropriate to the social context. 



The child uses cognitive skills to explore the environment, reason, and solve problems 
Child demonstrates an understanding of age-appropriate information. 

Child demonstrates recall of verbal and non-verbal events. 

Child understands and uses concepts related to early literacy and math skills. 

Child solves problems that require reasoning about objects, concepts, situations, and people. 



(Early Childhood Research Institute on Measuring Growth and Development, Draft Report, 1998) 
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CONSORTIUMS 



Several federal projects have involved the compilation of results from individual state 
efforts to develop family and child outcome measures for children with disabling or 
delaying conditions. Two such consortiums will be discussed in this section. By What 
Outcomes Should Part H Be Evaluated at the State Level? is the result of the gathering of 
state Part H program evaluators invited by the Subcommittee on Service Integration and 
Continuity of Services of the Federal Interagency Coordinating Council to address the 
above question. Longitudinal Studies of the Effects of Alternative Types of Early 
Intervention for Children with Disabilities: Follow-Up Institute is the final report 
produced by the Early Intervention Research Institute (EIRI) regarding a federally- 
sponsored longitudinal project that took place from 1990-1995. Summary results will be 
presented, and individual state projects will be discussed in Chapter 4 (Development of 
Family and Child Outcomes at the State Level). 

By What Outcomes Should Part H Be Evaluated at the State Level? 

May, 1997 

General Description 

The consortium focused on six main issues regarding the evaluation of Part H at the state 
level. These topics include the following: 1) service integration; 2) eligibility; 3) funding 
models and coordinating payment; 4) cost-effectiveness; 5) outcomes; and 6) measures. 
Since the focus of this chapter is on child and family outcomes, discussion will be 
restricted to the results regarding outcomes and measures. 

Ten states participated in the conference. The production of the final report was funded 
by the Maternal and Child Health Bureau, Health Resources and Services Administration, 
Department of Health and Human Services and written at the Early Intervention Research 
Institute at Utah State University. Participants were invited to two meetings in 
Washington D C. (August and November 1996) by the Federal Interagency Coordinating 
Council to discuss the results of Part H evaluation efforts by individual states. Most of the 
individual projects were funded by state legislatures or Part H programs. As a result, 
some states emphasized implementation or funding issues more than child and family 
outcomes, depending on the interests and requirements of the funding agency. Through 
discussions of participants, however, focus questions involving all of the six areas emerged 
as well as general recommendations. 
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CONSORTIUM PARTICIPANTS AND OUTCOME STATUS 



Participating States 


State Representatives 


Child 


Family 


Process 






Outcomes 


Outcomes 


Outcomes 


California 


Deborah Montgomery 




X 


X 


Colorado 


Heidi Phillips Shockley 


X 


X 


X 


Connecticut 


Mary Beth Bruder 


X 


X 


X 


Florida 


John Hall 




X 


X 


Hawaii 


Beppie Shapiro 


X 


X 


X 


Michigan 


Charito Melesky Hulleza 




X 


X 


New Jersey 


Steven Barnett 


X 


X 


X 


North Carolina 


Robin McWilliam 




X 


X 


Pennysylvania 


Jeffrey Brookfield 






X 


Utah 


Richard Roberts, Mark 
Innocenti, Linda Goetz 


X 


X 


X 



Family and Child Outcomes and Measures 

As the synthesis report states, “selection of specific outcomes is a difficult policy issue,” 
especially given the fact that the sole focus of the consortium was how to evaluate Part H 
specifically. While some states assumed that Part H evaluation would involve child and 
family outcomes achieved as a result of early intervention, others did not and focused 
specifically on program implementation evaluation (Roberts et al., 1997). 

States usually selected child and family outcomes with the counsel of experts, 
stakeholders, or both. A consensus of the consortium was that families and service 
providers should be involved in selecting the outcomes. With regard to measuring 
outcomes, it was noted that many outcome measures prevalent in early intervention 
evaluation cannot be reliably measured (see discussion in Chapter 7). Participants also 
stated that the nature of early intervention is to provide individualized services, which 
means that different outcomes may be used for measuring change in different children and 
families (Roberts et al., 1997). 



Within these general observations and concerns, participants indicated that family 
involvement in, and family satisfaction with, the service system are potential outcomes. 
Ways of measuring these outcomes include ascertaining whether a parent feels control in 
selecting services. Another prevalent family outcome was family support — do families feel 
the service system provides them with the support needed to nurture their child? 
Participants did warn, however, that past research has indicated that parents tend to report 
high levels of satisfaction regardless of service provision quality (Roberts et al., 1997). 
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Other recommendations focused on a family’s sense of improved quality of life as a 
function of receiving services. While quality of life is difficult to measure, participants 
suggested the following indicators: the ability of the family to participate in everyday 
community activities; adequate child care; parent ability to resume work; the increased 
manageability of child’s needs; parent opportunity to engage in life activities; and parent 
opportunity to socialize (Roberts et al., 1997). 

With regard to child outcomes, the consensus was that “developmental outcomes are 
important and Part H participation should afford positive effects in this domain.” While a 
clear consensus did not arise, several general suggestions in this area were made. 
Participants felt that specific developmental outcomes should be measured, as well as 
other possibilities such as child engagement with the environment, child persistence, and 
child temperament. Participants also suggested parent-child interaction and general health 
outcomes such as a more positive home environment, lower risk status for abuse, 
appropriate immunizations, appropriate medical care, appropriate child care, and evidence 
of a medical home (Roberts et al., 1997). 

Participants emphasized that a major factor in actually conducting studies which measure 
the above outcomes was cost: 

Most of the currently-funded evaluations use a mixed methodology that include 
interviews, questionnaires, and record reviews. Very few of the studies are 
conducting direct assessments of children and families and the direct measurement 
of program activities. The reasons these methods are not being used in more states 
are that direct assessment techniques are more expensive to implement (Roberts et 
al., 1997). 



MOST FREQUENTLY USED DATA 
SOURCES 

• interviews 

• questionnaires 
® record reviews 
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BY WHAT OUTCOMES SHOULD PART H BE EVALUATED: 
FAMILY OUTCOMES AND INDICATORS 



FAMILY OUTCOMES 


INDICATORS 


=> Family Involvement in Services 


parent feels control in selecting services 


=> Family Satisfaction 


parent feels control in selecting services 


=> Family Support 


parent feels service provides support 


=> Improved Quality of Life 


the ability of the family to participate in everyday 
community activities 

adequate child care 

parent ability to resume work 

the increased manageability of child’s needs 

parent opportunity to engage in life activities 

parent opportunity to socialize 



BY WHAT OUTCOMES SHOULD PART H BE EVALUATED: 
CHILD OUTCOMES AND INDICATORS 



CHILD OUTCOMES 


INDICATORS 


=> Developmental Skills 


unspecified 


=> Child Engagement with the Environment 


unspecified 


=> Child Persistence 


unspecified 


=> Child Temperament 


unspecified 


=> Parent/Child Interaction 


unspecified 


=> General Health 


more positive home environment 
lower risk status for abuse 
appropriate immunizations 
appropriate medical care 
appropriate child care 
evidence of a medical home 
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Some general concerns expressed regarding data collection methods mentioned in the 
table were: Likert scales of satisfaction are inadequate in examining satisfaction; there are 
issues with report reliability from parents/providers; pre-existing state data bases are often 
not consistent year-to-year and merit caution; and questionnaires whose distribution relied 
on community service providers were often given only to those favorably disposed to that 
service provider (Roberts et al., 1997). 

General conclusions regarding outcomes and measures were as follows: 

• Few outcome measures have been developed to the point where they could be 
used to assess the results of a comprehensive, coordinated system of services for 
families. 

• Discovery of such outcomes is just beginning. 

• Most current evaluation relies on implementation assessment. 

• To what degree emphasis should be placed on particular development outcomes as 
opposed to well being, inclusion, and social adaptation is heavily debated. 

• Measurement issues are very problematic so multiple measures should be used 
(Roberts et al , 1997). 

EARLY INTERVENTION RESEARCH INSTITUTE (EIRI) 

1986-1995 

General Description 

The Longitudinal Studies of the Effects of Alternative Types of Early Intervention for 
Children with Disabilities is a synthesis report of a series of longitudinal studies which 
took place from 1986-1995. The original project began enrolling subjects in sixteen 
different studies in October 1986. These sixteen studies collected data until the fall of 
1990. Nine of the original sixteen studies conducted follow-up research from 1990-1995. 
This report summarizes findings for the nine studies followed longitudinally and includes 
data for the entire span of the projects, 1986-1995 (Innocenti, 1996). Only seven of these 
studies have been included in this discussion as two of the studies applied to preschool 
children rather than those aged birth to three. 

The studies were funded by the U S. Department of Education in order to evaluate the 
effects and costs of alternative types of early intervention programs for children with 
disabilities. The Department of Education sought to address three factors and concerns of 
the time (1985): 1) the limited high quality early intervention research for children with 
disabilities; 2) pressures to expand early intervention programs for children with 
disabilities; and 3) the almost total absence of efficacy research which included a cost- 
benefit analysis. As a result it was required that each study 1) be a true experiment 
(randomized controlled trial) in which two alternative types of intervention were 
compared; 2) consider the effects of the intervention on both children and families; 3) 
analyze the cost of the alternative types of intervention in conjunction with the effects; and 
4) be carried out in field-based settings which were representative of the state-of-the-art 
early intervention programs. The three elements of early intervention to be examined were 
intensity, age at start of intervention, and program variation. The general aim of the 
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studies was to ascertain which elements of early intervention were working well and at 
what cost (Innocenti, 1996). 

EIRI: PARTICIPATING SITES AND RESEARCH FOCUS 



PARTICIPATING SITE 


RESEARCH FOCUS 


New Orleans, Louisiana 


Intensity 


Chicago, Illinois 


Intensity 


Bentonville and Fayetteville, Arkansas 


Intensity 


Salt Lake City, Utah 


Age at Start 


Charleston, South Carolina 


Age at Start 


Columbus, Ohio 


Program Variation (coordinated versus 
traditional NICU transition services) 


Des Moines, Iowa 


Program Variation (parent involvement) 



Child and Family Outcomes 

The summary report emphasized that the selection of child and family outcomes was 
restricted because “the procedures for assessing child change attributable to intervention 
are inadequately developed, and totally satisfactory instruments are not available. With 
regard to measuring the impact of early intervention on families, the field is even less well 
developed.” In response to concerns that early intervention research has been too 
narrowly focused on intelligence measures, other areas such as social-emotional growth, 
adaptive behavior, and elements of family functioning were included. Participants 
indicated that although it was desirable to include broader outcomes, lack of 
instrumentation and cost were factors (Innocenti, 1996). 

Within this framework, all studies included the outcome domains of child and family 
functioning. Specific outcomes included the following: cognitive, motor and language 
skills; mother and child interaction; parental attitudes towards their child with a disability; 
special education placement and grade retention; and parent stress. Each participating 
study site had to include the core set of assessment instruments (listed in the table below), 
but could include other measures to suit specific needs. 




25 



EIRI: CHILD AND FAMILY OUTCOMES AND INSTRUMENTS 



CHILD OUTCOMES 


FAMILY 

OUTCOMES 


INSTRUMENTS 


• Social Skills 

• Adaptive Skills 

• Motor Skills 

• Communication Skills 

• Cognitive Skills 

• Broad Knowledge 

• Personal Living 

• Community Living 

• Independence 

® Problem Behaviors 

• Academic Competence 

• Child Health 


• Adaptability 

• Cohesion 

• Social Support 

• Stress 


CHILD 

Battelle Development Inventory 
Woodcock-Johnson Tests of 
Achievement 

Scales of Independent Behavior 
Social Skills Rating Scale 
Pictorial Scale of Perceived 
Competence and 
Acceptance 

FAMILY 

Parenting Stress Index 
Family Support Scale 
Family Resources Scale 
Family Inventory of Life 
Events and Changes 
Family Adaptation and 
Cohesion Evaluation 
Scale-Ill 
Child Health 
Additional Services 
Family Information Survey 



While this report brought together a number of well-designed research projects, it should 
be noted that the results cannot be generalized to the national population. The 
participation of state sites was assessed on specified criteria, and the state sites themselves 
were not representative of statewide programs or children and families with disabling or 
delaying conditions. 

SUMMARY 

Review of the above projects presents several implications for the state of Washington. In 
selecting child and family outcomes for children with disabling or delaying conditions and 
their families several factors should be taken into account: 1) purpose of study; 2) type of 
information desired; 3) costs. Often factors one and two are influenced by factor three. 

Purpose of Study 

It is important to be able to specify precisely what information is desired. If a state wishes 
to examine directly early intervention impacts on children and families with individual 
observations or use standardized tests, then the cost rises. Most studies of this nature are 
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conducted by universities with federal grant money on samples of children with specific 
participation criteria. If the purpose of the study is changed to examining the perception 
of various participants regarding early intervention rather than attempting to directly 
measure outcomes, a state could use non-standardized instruments such as surveys. Given 
the measurement difficulties, inadequacies of standardized tests for the disabled and 
delayed condition population, the difficulty of selecting pertinent outcomes, and the costs, 
many states have chosen evaluation studies that do not assess directly child and family 
outcomes. While much state research activity does attempt to assess directly family and 
child outcomes, the purpose of such studies is usually not to assess early intervention 
broadly, but to examine a population affected by a specific condition (such as autism) or 
intervention feature (such as intensity). 

Type of Information 

If the type of information desired is to be obtained by non-standardized methods, 
outcomes should be stated to reflect the methodology. For instance, in the National Early 
Intervention Longitudinal Study (NEILS), where outcomes will be assessed primarily by 
surveys and interviews, outcomes were worded to reflect a subjective approach. For 
example, the outcome was stated, Did early intervention enhance the family ’ s perceived 
quality of life? Rather than the more direct, Did early intervention enhance the family’s 
quality of life? 

If the study chooses standardized tests, gains in standardization may be diminished by lack 
of instrument sensitivity. As was mentioned in the Early Intervention Research Institute 
(EIRI) study, researchers were frustrated that many outcomes of interest, such as child- 
parent interaction and social skills, had few reliable instruments (especially for the disabled 
and delayed condition population). Scientific studies have often relied on measures of 
intelligence simply because the instruments are, for the most part, reliable. Researchers 
question, however, whether a small gain in IQ is as important an outcome as broader 
social ones, like, “the child is able to play appropriately with others.” 

The Early Childhood Research Institute on Measuring Growth and Development Study 
(ECRI-MGD) offers promise in regard to this dilemma. The selection of child outcomes 
acceptable to early intervention stakeholders that can be traced from birth to eight years 
and incorporate both technical skills and social ones is an encouraging development. The 
issue yet to be addressed is whether reliable objective measurement instruments can be 
developed for the outcomes. Individual national groups are also making efforts in this 
area. The American Speech-Language Association, for instance, is working on a National 
Outcomes Measurement System (NOMS) for speech-language pathology and audiology. 
The system is designed to develop a national outcomes database to demonstrate the value 
of speech-language pathology and audiology services. NOMS includes four components, 
each targeted to specific populations: birth to kindergarten; school-aged in education; 
school-aged in health care settings; and adults (Moore, 1998). 
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Costs 



As described above in the review of the consortium on state efforts to evaluate Part H, 
most projects relied on information collected via surveys, record reviews, or 
questionnaires. Participants expressed that standardized tests are expensive, not 
standardized to children with disabling or delaying conditions, and in some cases not 
reliable measures of the outcome at hand, especially outcomes other than child 
intelligence. Non-standardized instruments can also be tailored to suit individual programs 
and information requirements. Such measures, however, have not been validated as 
extensively as many standardized tests and offer little objective assessment of family and 
child outcomes. 

Another element of cost to consider is the number of study subjects needed in order to 
gain statistically significant results that can be generalized to the state population (of 
children with disabling or delaying conditions and their families). Usually these numbers 
are quite large, which increase research costs. As a result, often objective evaluation of 
family and child outcomes involves a smaller sample population that cannot be used to 
generalize the results statewide, but can draw some conclusions about early intervention 
for a specific population (e g. autistic children) or program element (e g. intensity). 

Decision makers in Washington State, as other states, should take into consideration these 
factors when deliberating which type of study to conduct. 
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CHAPTER 4 



DEVELOPMENT OF FAMILY AND CHILD OUTCOME 
MEASURES AT THE STATE LEVEL 



INTRODUCTION 

State activity takes place on several levels. Some state studies focus on Part H 
implementation; some focus on child/family outcomes research which sample from a 
defined locale within the state; and others examine study feasibility issues. Studies are 
usually conducted by state agencies, universities, independent research organizations, or 
programs themselves. This section describes various state research efforts which were 
presented at the National Early Childhood Technical Assistance System (NEC*TAS) 
Conference in 1997, at the Part H Consortium, or were part of the Early Intervention 
Research Institute study (see Chapter 3, Development of Family and Child Outcome 
Measures at the Federal Level, for descriptions). Only those directly relevant to child and 
family outcomes will be discussed in detail. 

The following table summarizes the activity of those states which presented at the 
NEC*TAS conference. The three categories of study types are implementation, research, 
feasibility/exploratory. Implementation studies assess Part H and Part H programs. All 
include process outcomes, most include family outcomes, and some include child 
outcomes. Research studies assess the impact of a particular aspect of early intervention 
(such as program intensity) by examining selected child and family outcomes. 
Feasibility/Exploratory studies focus on investigating assessment options and gathering 
program information and participant input. 
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SUMMARY OF BIRTH TO THREE EARLY INTERVENTION 
STATE RESEARCH PROJECTS PRESENTED AT NEC*TAS 



State 


Type of Study 


Child 

Outcomes 


Family 

Outcomes 


Process 

Outcomes 


Arkansas 


Research 


X 


X 




California 


Implementation 




X 


X 


Colorado 


Implementation 


X 


X 


X 


Connecticut 


Research 


X 


X 




Delaware 


Research 


X 


X 




Florida 


Implementation 




X 


X 


Hawaii 


Feasibility/ 


X 


X 


X 




Exploratory 








Illinois 


Research 


X 


X 




Indiana 


Feasibility/ 










Exploratory 








Louisiana 


Research 


X 


X 




Michigan 


Implementation 




X 


X 


New Jersey 


Implementation 


X 


X 


X 


North Carolina 


Implementation 






X 


Ohio 


Research 


X 


X 




South Carolina 


Research 


X 


X 




Utah 


Research 


X 


X 


X 


(2 studies) 


Implementation 


X 


X 


1 
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IMPLEMENTATION STUDIES 



The goal of implementation studies is to assess Part H. All of these studies include 
process outcomes, most include family outcomes, and some include child outcomes. As 
the focus on this section is child and family outcomes, process measures will not be 
discussed. When reviewing child and family outcomes, it becomes necessary to discuss 
how the outcomes will be measured. The following table lists the child and family 
outcome measures, as well as the data source, selected by those states presenting at the 
NEC*TAS conference who were conducting implementation studies. 



CHILD AND FAMILY OUTCOMES AND DATA SOURCES: 
IMPLEMENTATION STUDIES (NEC*TAS) 



State 


Child Outcomes 


Family Outcomes 


Data Sources 


California 




Family Satisfaction 


Family Survey 


Colorado 


Infants Enhance 
Development 


Parents Gain Support 

Families Understand 
PartH 


Focus Groups With Parents 
Interviews With Providers 
Provider Survey 
Family Survey 
Community Self-assessment 
Record Reviews 
Site Visits 


Florida 




Perception of 
Effectiveness 

Parent Satisfaction 


Family Written Sur/ey 
Family Interview 


Michigan 




Increased Support 

Empowerment 

Stress 


Part H Coordinator Survey 
Provider Survey 
Family Survey 
Site Visits 


New 

Jersey 


Child 

Development 
Child Behavior 


Parenting Stress 

Mother/Child 

Interaction 


Standardized Tests 
Records Review 
Family Interview 
Service Provider Survey 


Utah 


Child 

Development 

Transition 

Experience 


Family Satisfaction 
Participation in IFSPs 
Transition Experience 


Standardized Test 
Parent Interviews 
Teacher Interviews 
Family Questionnaires 
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The most common child outcome selected was child development. In the family outcome 
domain, parent satisfaction, support and empowerment outcomes were most frequent. 
Methods of data collection relied heavily on surveys and interviews. Standardized tests 
were used to measure general child development. 

RESEARCH STUDIES 

The following tables describe state projects that participated in the Federal Early 
Intervention Research Institute (EERI) project from 1986-1995 (see Chapter 3, 
Development of Family and Child Outcome Measures at the Federal Level, for a 
description). All projects were randomized controlled studies in which two or more 
alternative types of intervention were compared. 

EIRI STATE RESEARCH PROJECTS: TOPICS AND SUBJECT NUMBERS 



LOCATION 


STUDY TOPIC 


NUMBER 

OF 

SUBJECTS 


Arkansas (Bentonville 
and Fayetteville) 


Standard intensity compared to two variations of 
intensity 


78 


Illinois (Chicago) 


Once a week services compared to three times per 
week services 


72 


Iowa (Des Moines) 


Classroom intervention with parent involvement 
compared to classroom intervention without 


76 


Louisiana (New Orleans) 


Weekly, individual parent-child sessions compared 
to monthly parent group meetings 


35 


Ohio (Columbus) 


Coordinated, comprehensive service for medically 
fragile infants prior to discharge compared to 
routine service 


50 


South Carolina 
(Charleston) 


Compared 3 months adjusted age at start for 
medically fragile premature infants to 12 months 
adjusted age at start 


68 


Utah (Salt Lake City) 


Compared 3 months age at start for medically 
fragile infants to 1 8 months age at start 


60 
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Some outcomes were common to all states, and some were state specific. All of the 
research studies included the outcome domain of child development. Indicators of child 
development involve the following: Social Skills, Adaptive Skills, Motor Skills, 
Communication Skills, Cognitive Skills, Broad Knowledge, Personal Living, Community 
Living, and Independence. 

Other child outcomes included in all the studies were Academic Competence and Child 
Health. Family outcomes which were included in all the studies were adaptability, 
cohesion, social support, and stress. All outcomes were measured by standardized tests. 
The following table displays those outcomes which each state additionally chose to add to 
the study designs. 

EIRI: ADDITIONAL CHILD AND FAMILY OUTCOMES 
SELECTED BY PARTICIPATING STATES 



Location 


Child Outcomes 


Family Outcomes 


Data Sources 


Arkansas 


Child Self-concept 


Parent Self-awareness 
Parent Participation 
Feedback About Intervention 


Standardized Tests 
Questionnaire 


Illinois 


(Requirements Only) 


(Requirements Only) 


Standardized Tests 


Iowa 


(Requirements Only) 


Knowledge of Parents 


Standardized Test 


Louisiana 


(Requirements Only) 


Mother/Child Interaction 

Parental Attitudes Towards 
Child 


Standardized Tests 


Ohio 


Child Temperament 


Parent Protective Behavior 
Parent Vulnerability 


Standardized Tests 


South 

Carolina 


Child Self-concept 


(Requirements Only) 


Standardized Tests 


Utah 


Child Temperament 


Parent/Child Interaction 
Parent Self-concept 
Parent Participation 


Standardized Tests 



The research studies used almost exclusively standardized tests and had small subject 
numbers. Results could not be generalized to the state’s population of children with 
disabling or delaying conditions and their families because subject selection took place 
within a specific locality. The purpose of these studies was to assess the impact of certain 
elements of early intervention rather than generally assess early intervention in the state. 




• 33 



42 



FEASIBILITY /EXPLORATORY STUDIES 

Some states have chosen to conduct feasibility studies which assess methods for 
evaluating early intervention. As with other types of studies, feasibility projects can be 
either narrowly focused (e g. How do we evaluate Part H implementation?) or more 
broadly construed (How do we evaluate early intervention services?). Exploratory studies 
often seek to gain understanding of program operations, ascertain perceptions of early 
intervention participants, or identify strengths and needs. The following is a description of 
each type of study conducted by two states participating in the NEC*TAS conference. 

Feasibility — Hawaii 

The University of Hawaii received funds from the Office of Special Education Programs, 
U.S. Department of Education to conduct two feasibility studies (1992-1995) which 
addressed the following questions: 

1) How effective and efficient is the current early intervention system? 

2) What outcomes are achieved by families through early intervention? 

3) What outcomes are achieved by children through early intervention? 

The discussion of the studies in this section will focus on child and family outcomes. Each 
of the studies was a two-year project. Data collection methods involved a literature 
review, key informant (parents, service providers, program coordinators, medical 
personnel) interviews and surveys, focus groups, site visits, records review, and checklists. 

What Outcomes Are Achieved by Families Through Early Intervention? 

The major finding of this study was the lack of information available in early intervention 
programs regarding identification of the needs (and strengths) of enrolled early 
intervention families in Hawaii. The research team first tried to identify family needs and 
how they were being addressed in order to assess what family outcomes should be. Since 
information regarding needs was lacking, the study concentrated on creating 
methodologies for collecting this information. As a result, recommendations regarding 
actual outcomes were limited by the fact that information on actual goals and objectives 
for families was not present. The study did recommend, however, two broad domains: 
empowerment and family functioning. 

What Outcomes Are Achieved by Children Through Early Intervention? 

The major recommendation of this study in regard to assessment was to conduct a series 
of small-scale evaluations which focused on the two child outcomes of life skills and 
school readiness. In terms of evaluation methods, the study concluded that “the only 
evaluations which seemed feasible were non-experimental designs without comparison 
groups.” (Roberts, 1997). It was suggested that school readiness could be measured by 
asking kindergarten teachers to rate random samples of early intervention “graduates” and 



compare them to those who had not received early intervention (Personal E-mail 
Communication, Beppie Shapiro, project coordinator, 4/6/98). 

In summary, the research team concluded that “it is impossible at the moment to evaluate 
early intervention programs by measuring child developmental outcomes. The complexity 
of adjusting achieved developmental status, at say, three years, to account for the 
difference between expected gains given child diagnosis, type and degree of environmental 
risk, severity or degree of developmental problem, and maturity, is overwhelming. Since 
Part H is an entitlement, no comparison group can exist.” (Personal E-mail 
Communication, Beppie Shapiro, project coordinator, 1/6/98). It was suggested that 
focusing on process measures (effectiveness of Child Find, timeliness of service, 
accomplishment of IFSP goals) was a more feasible approach to early intervention 
evaluation for Hawaii. 



CHILD OUTCOME DOMAIN 


FAMILY OUTCOME DOMAIN 


Life Skills 


Empowerment 


School Readiness 


Family Functioning 



Exploratory Study — Indiana 

The Center for Innovative Practices for Young Children, an affiliated program with 
Indiana University-Bloomington, is conducting a qualitative study examining Indiana’s 
First Steps Early Intervention System (FSEIS) for the Bureau of Child Development in 
Indiana’s Family and Social Services Administration. The purpose of the study is to 
accomplish the following: 

1) to determine expressed purposes, goals, and guiding principles of FSEIS in order 
to determine a set of criteria for evaluating quality and impact of early intervention 
services; 

2) to discover how services are carried out; determine the impact of FSEIS on 
children, families and communities as determined by early intervention 
stakeholders; 

3) to determine perception of stakeholders with regard to program quality; 

4) to determine needed resources and supports. 

The study took place from 1995-1997 and the final report is pending at this time. In 
exploring the above issues the principal evaluators examined data from existing databases 
and other evaluation activities, conducted telephone interviews with stakeholders, 
conducted on-site interviews and observations, conducted family interviews, surveyed 
families statewide, and established a State Review Team (www.isdd.indiana.edu/ 
fseval.html, 3/17/98). About 159 stakeholders participated from around the state 
(Personal E-mail Communication, Michael Conn-Powers, researcher, 4/7/98). 



While the final report is pending, one major finding of the study was the “absence of 
clearly defined outcome-oriented goals among constituents.” Participants emphasized 
process and service quality oriented goals. In regard to child outcomes, improvement in 
child development and the acquisition of major milestones were common themes. 
Researchers pointed out that selecting specific milestones and skills was problematic due 
to the diversity of needs and strengths in the early intervention population. It is under 
consideration to recommend an “exiting” measurement survey which would ask families 
what their goals were in this area and how well they accomplished them. (Personal E-mail 
Communication, Michael Conn-Powers, researcher, 4/7/98) 

SUMMARY 

In terms of the statewide evaluations discussed above, Child Development, Readiness to 
Learn, and Life Skills are common child outcomes. Such outcomes can be measured both 
through standardized tests, or non-standardized data collection methods (non-standardized 
surveys, interviews, questionnaires). Family Empowerment and Family Functioning are 
prevalent family outcome domains which can also be measured by standardized or non- 
standardized instruments. Indicators for these domains are quite varied and can depend on 
the data collection method and the focus of the study. 

Evaluation of early intervention programs with regard to family and child outcomes is a 
complicated matter. Issues of cost, scientific validity, study designs and focus are a few 
common topics emerging from the summary of state activity by NEC*TAS, Early 
Intervention Research Institute study and Consortium participants. Some states have 
chosen to evaluate early intervention programs in terms of process measures. Some have 
chosen child and family outcomes that are suited to subjective evaluation (such as 
satisfaction or perceived impact); others have selected projects which use more objective 
means to measure outcomes and focus on specific aspects of early intervention programs. 
Others have conducted exploratory or feasibility studies to collect decision-making 
information. In choosing which studies to do, states should balance cost, scientific 
validity, desired information, and desired study focus in order to determine what is best 
suited to their particular circumstances. 
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CHAPTER 5 



LOCAL ASSESSMENT IN WASHINGTON STATE 

A number of communities and programs in Washington State have developed procedures 
for assessing local early intervention efforts. The Local Assessment Survey (Appendix D) 
was developed to identify program evaluation activities undertaken at the local level that 
might inform statewide evaluation activities. The survey was sent to child developmental 
programs. County Interagency Coordinating Councils, Educational Service Districts, 

Infant Toddler Early Intervention Program Lead Agencies and additional contractors, 
neurodevelopmental centers, school district Special Education Directors for schools who 
currently serve children, tribal authorities, University Affiliated Programs, and University 
Special Education Teacher Preparation Programs. 

A total of 309 surveys were mailed. This included duplicate listings for organizations both 
within and between targeted mailing categories. With only a few exceptions, no attempt 
was made to contact non-responders after a single mailing. As this was intended to be a 
probe for information, responses were not anticipated from all those surveyed. Eighty- 
nine organizations responded, from programs within all of the categories listed above and 
associated with every county. 

LOCAL ASSESSMENT ACTIVITIES 



Responses from the Local Assessment Survey were focused into three categories: 

1) surveys developed by an organization to provide specific feedback; 2) processes used to 
track children ’ s progress ; and 3) tracking children ’s services after age three. 

Surveys 

All programs that contract with the Infant Toddler Early Intervention Program receive its 
Self-Assessment Tool (Infant Toddler Early Intervention Program). This includes sample 
parent/family and provider surveys, used directly by some organizations. Other programs 
have developed their own surveys. Thirty-four Local Assessment Survey respondents 
indicated that they had developed their own survey(s) targeted at either families and/or 
staff. These respondents included County Interagency Coordinating Councils, 
developmental centers, neurodevelopmental centers, school districts, Infant Toddler Early 
Intervention Program contractors, tribal organizations, and University programs. The 
represented organizations reside in eighteen counties and are spread among counties 
which are metro (38%), small urban (35%) and rural (26%) in character. 



Twenty-six respondents had developed parent/family surveys which addressed issues such 
as whether families perceived their needs as being met; program convenience, 
responsiveness, and/or sensitivity; coordination of services; family choice, awareness of 
options, and/or control; level of comfort with staff and/or program; knowledge gained; the 
referral experience; specific staff areas. Family Resources Coordinators; level of 
involvement or participation; and family support. Some surveys also discussed obstacles, 
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life improvements, and stress levels. An additional ten surveys were directed toward 
families but were more specific in nature, i.e., concerned with parent needs, health, 
interests, gross child development, the fit of the parent match (for parent support), 
transportation issues, or transitions. 

Four respondents mentioned staff surveys. These included a self-assessment for “helping” 
parents (parents involved in parent support), a survey which addressed family support 
worker duties, and staff surveys concerned with transition. 

Tracking progress 

All programs track the individual progress of children through their Individualized Family 
Service Plans (IFSP). These plans outline service components and desired outcomes for 
children and their families, and are updated periodically to reflect new goals as 
development and needs are reassessed. While many respondents cited this process of 
tracking children’s progress, thirteen respondents noted specific additional or related 
tracking activities. Represented organizations include developmental centers, 
neurodevelopmental centers, school districts, Infant Toddler Early Intervention Program 
contractors, tribal organizations, and University programs, and are located in twelve 
counties which are metro (27%), small urban (3 1%) and rural (42%) in character. 

Of these organizations, almost half (six) noted the use of various curriculum-based 
assessment, evaluation, and programming systems, which track progress and help with 
individual goal-setting. Other responses included broad compilations of numbers served 
and exited, as well as more specific programs such as a monitoring program designed for 
high school parents and their children. 

The Neurodevelopmental Centers of Washington have begun an additional tracking 
activity, partially supported by the Department of Health. Six neurodevelopmental centers 
are taking part in a one-year trial of WeeFIM, a functional improvement measurement tool 
for children from six months to seven years of age and older. This is the pediatric version 
of the adult Functional Improvement Indicator (FIM) for patients undergoing medical 
rehabilitation. As with FIM, a national database of subscribers’ data allows users to 
compare their data with regional and national performance figures. Information collected 
includes patient demographic characteristics, medical diagnoses, impairment groups, 
lengths of inpatient stay, costs, and functional assessments (self-care, sphincter control, 
transfers, locomotion, communication, and social cognition) undertaken during admission, 
treatment, discharge, and follow-up. The Neurodevelopmental Centers are working with 
Uniform Data System for Medical Rehabilitation (developers of WeeFIM) to increase the 
sensitivity of WeeFIM for the birth to three population. At the end of the trial period 
(October 1998) the Neurodevelopmental Centers of Washington will evaluate the 
applicability and usefulness of WeeFIM. 
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Tracking services after age three 

A number of school district programs which offer both birth to three early intervention 
services (Part H) and preschool services (Part B) pointed out that they track children who 
remain eligible until kindergarten. In addition, five organizations mentioned longer term 
tracking activities. Sedro-Woolley School District noted that through the Washington 
State Information Processing Cooperative, they are able to track special education 
children from eligibility to the time of exit. Bethel School District tracks those who 
transition into district services, and is exploring the possibility of tracking through grades 
three or four. Two tribal organizations (the Roger Saux Health Center of the Quinault 
Indian Nation and the Port Gamble S’Klallam Tribe) noted that they have computerized 
systems by which they track their children, and this continues after age three. The Little 
Red School House, a developmental center with sites in Lynnwood and Everett currently 
serving approximately 130 children, is beginning to survey its graduates in order to gain 
insights into families’ early intervention experiences and to track later placement and 
quality of life. They plan to routinely track new graduates at preschool entry and at the 
start of kindergarten, with more random sampling at later points in time. 

RESEARCH STUDIES 

Several more extensive research studies related to early intervention for infants and 
toddlers, birth to three, were identified within Washington State. 

Washington Research Institute 

Supporting Neurobehavioral Organizational Development in Infants with Disabilities: 
The Neurobehavioral Curriculum for Early Intervention. Rodd Hedlund, Principal 
Investigator, Washington Research Institute. The purpose of this project is the 
development, field-testing, and evaluation of a neurobehavioral curriculum for infants bom 
with very low birthweight and their parents. The model assists early intervention 
professionals to: 1) observe and interpret the behavioral cues expressed by the infant; 2) 
learn and implement specific neurobehavioral strategies that support the infant’s 
neurobehavioral organization and self-regulatory competence; and 3) support parents as 
well as paraprofessionals to recognize and support the neurobehavioral needs of these 
fragile infants. 

A comprehensive evaluation of the Neurobehavioral Curriculum for Early Intervention 
(NCEI) was conducted for formative and summative purposes using both quantitative and 
qualitative procedures to capture the impact of the curriculum on a variety of indices 
including standardized measures of infant development, subtle changes in infant behaviors, 
and subjective impressions and feelings of care givers and early intervention professionals 
and paraprofessionals. Fifteen early intervention professional and paraprofessionals from 
three sites (Tacoma Learning Center, Valley Learning Center, and the Hospital to Home 
Program) participated in the field-testing and evaluation of the NCEI. Data were 
collected on a total of 48 infants (26 male and 22 female) and their families. To be 
included in the project, infants had to be chronologically or developmentally six months of 
age or younger. The infants’ average chronological age was 1 1 months, ranging from 2.5 



to 3 1 months. Average corrected age was 9 months, ranging from birth to 27 months. 

The infants’ average gestational age was 33.5 weeks and their average birth weight was 
2062 grams (~4.5 lbs). Forty of these infants had three or more significant medical 
conditions such as prematurity and very low birthweight (<1500 grams, 3.3 lbs), broncho- 
pulmonary dysplasia, developmental delays and vision or hearing impairments. 

Results 



The field testing and evaluation focused On five main areas: impact of the curriculum, 
impact of training, fidelity of implementation, formative evaluation of materials, user 
satisfaction, and cost-effectiveness. 

Infant progress and caregiver-infant interactions 

Pre- and post-test data were collected on infants’ progress using the Bayley Scales of 
Infant Development and the Early Intervention Developmental Profile. On the Bayley 
Scales, significant gains (p < .05) were found for both mental and the psychomotor age 
equivalencies. On the EIDP, significant gains (p<05) were found for age equivalencies in 
all six domains. Cognitive, Gross Motor, Communication, Fine Motor, Self-help, and 
Social. Data on infants’ progress on I ESP goals and objectives (outcomes) and degrees 
and amount of neurobehavioral supports were collected on a quarterly basis. Over a six 
month intervention period, infants’ progress on IFSP goals was significant for all six 
developmental domains. Decreases over time in degree and amount of neurobehavioral 
facilitation were also significant for cognitive, gross motor, communication, and fine 
motor goals. 

Overall these results show that infants made significant progress on their IFSP goals and 
objectives (outcomes) as well as decreased their need for neurobehavioral facilitation. 

Significant improvement in neurobehavioral organization was also found using the Infant 
Behavioral Assessment (IBA), reflecting greater infant competence and reduced stress 
during interactions. 

Qualitative information gathered from the early interventionists through questionnaires, 
informal interviews, and direct observations confirmed that interventionists found the 
NCEI strategies very helpful in improving infants’ neurobehavioral organization and had 
an impact on infants’ gains on their IFSP goals (outcomes). 

Interventionists also found that as a result of their use of the NCEI strategies and parent 
materials, caregivers appeared more comfortable in a number of ways with their infants. 

Impact of training, fidelity of implementation and formative evaluation of materials 

Overall, trainees found the IBA and NCEI training very effective in increasing their 
knowledge on infant neurobehavioral organization, their ability to interpret infants’ cues 
and provide neurobehavioral support to infants and families. Most interventionists 
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implemented the NCEI with all or most of the infants and families in their case load, 
including older children who were functioning at a low level. 

The NCEI and the parent materials were found to be well thought out and organized. The 
neurobehavioral strategies and activities were comprehensive and easily understood. The 
parent materials were found to be positively worded, and understandable for average level 
readers. 

User satisfaction and cost effectiveness 

Overall, interventionists expressed satisfaction with most of the strategies and materials, 
and felt that parents also generally liked the strategies and materials. Most interventionists 
thought the strategies and parent materials were very easy to integrate within daily 
activities, and that they were very reasonable in terms of time and effort for preparation 
and implementation. 

University of Washington 

The Early Child Care Study of Children with Special Needs. Cathryn Booth, Principal 
Investigator, and Jean Kelly, Co-Principal Investigator. This longitudinal, ecological study 
is being conducted through the Department of Family and Child Nursing at the University 
of Washington. This project is designed to evaluate the effects of child care on the 
development of young children with special needs. The sample (n=160) includes those 
with diagnosed disabilities or delays (n=80) and those at risk for developing a delay 
(n=80) due to prematurity, respiratory distress syndrome, maternal abuse of drugs or 
alcohol, or other specified factors. 

A longitudinal design is being used to evaluate child outcomes at 30 and 45 months as a 
function of the complex interactions among child characteristics, family characteristics, 
and the quality and type of home, early intervention, and child care environments in which 
the children develop. Children and their families participating in the study are being 
assessed at child ages 12, 15, 30 and 45 months. Interviews, questionnaires, structured 
and unstructured observations occur at home, at child care, and in the laboratory setting. 

Families were recruited when the children were 12 months of age and are being followed 
through 45 months. A grant has been submitted to follow the children through second 
grade, in order to look at transition, inclusion and school-age outcomes. 

Training and Early Intervention to Promote Quality Parent-Child Interaction in 
Homeless Families. Jean Kelly, Principal Investigator. This study is being conducted 
through the Department of Family and Child Nursing at the University of Washington. 

Background and Purpose of Research 

This project will provide early intervention services to homeless families being served by 
the YWCA, and train YWCA staff to provide the intervention services after the project is 
completed. One of the terrible aspects of family homelessness is a dramatically increased 
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risk that homeless children will be impaired both psychologically and cognitively, with 
immediate and long-term consequences for these children and society. The focus of this 
intervention will be to facilitate nurturing interactions between homeless parents and their 
children, birth to three years of age, in order to mitigate the children’s risk for 
psychological problems and delayed development. The specific objectives of the study 
are: 

1 . To provide training to Greater Seattle YWCA program sites serving homeless 
families on how to provide one-on-one early intervention to facilitate healthy 
parent-child interactions. 

2. To provide direct early intervention with parents and their young children, birth 
to three years of age, who are in YWCA transitional housing. The early 
intervention sessions will be the basis of the training. The intervention is designed 
to increase the quality of parent-child interactions. 

3. To evaluate the project to determine changes in (a) the quality of interaction 
between participating parents and children and (b) YWCA parent educators’ 
knowledge, performance, and satisfaction. 

Evaluation 

The pre- and post-assessments of staff (videotaped sessions of educators with parent-child 
dyads and knowledge and satisfaction questionnaires), and the parent-child interactions 
(videotaped sessions) will be used to evaluate the success of this training and early 
intervention project. Coders, blind to the design of the project, will code the videotapes of 
the parent-child play and teaching episodes, and the videotapes of the parent-educator 
sessions. The parent-child interactions will be coded using the Nursing Child Assessment 
Teaching Scale (NCATS) (Barnard, 1994). The parent-educator interactions will be 
coded using a coding scheme developed for the purposes of the study. Additionally, the 
parent-educator will complete The Competency Rating Scale (adapted from Leitch, 1995) 
used to rate knowledge and skill in each of the following areas: knowledge of 
infant/toddler states, behaviors and cues, parent-child interaction, providing feedback to 
parents during interactions, and establishing helping adult relationships in the early 
intervention settings, and a staff satisfaction questionnaire. 

Parent-Child Communication Coaching. Kathryn Barnard, Principal Investigator, Susan 
Spieker and Colleen Morisset Huebner, Co-Investigators. This study, which is part of a 
national consortium of 15 projects evaluating the Early Head Start program, is being 
conducted through the University of Washington, School of Nursing, in collaboration with 
Families First Early Head Start program of the Children’s Home Society of South King 
County. This project is designed to measure outcomes of an intervention known as 
Parent-Child Communication Coaching, whose intent is to facilitate the parent-child 
interaction in a positive manner and to facilitate the child’s secure attachment to the 
parent, thereby boosting the child’s cognitive and language development and preventing 
learning and emotional problems. 



Two hundred families enrolled in the Families First program will be studied. One hundred 
of the families will receive the intervention, and 100 will serve as a control group. The 
core service of the intervention is a home visiting program. The Parent-Child 
Communication Coaching begins in pregnancy and continues with age specific 
interventions through the second year. 

The broader Early Head Start program goals, which each have specific outcomes, include: 

1 . Enhance child development and social competence by promoting the child’s 
development and supporting the provision of developmentally enriching 
caregiving and environments in the family’s home and, when appropriate, the 
child care center. 

2. Support family development and promote long term self-sufficiency and 
capability of parents. 

3. Create stronger, responsive communities that share responsibility for the 
healthy development of children. 

4. Recruit, develop and retain high quality staff. 

Snohomish County 

Snohomish County and the Region III office of the Division of Developmental Disabilities 
recently worked with consultant Susan Janko, Ph.D., University of Washington College of 
Education, to conduct a review of birth to three services for children with disabilities in 
Snohomish County. A community steering committee including families, educators, and 
service providers helped direct and advise the study. 

Data were collected through the use of interviews, observations, and review of program 
documents. Interviews were conducted with families of children currently using birth to 
three services, families of children who had previously used birth to three services, families 
of children who were eligible for but did not use birth to three services, and 
representatives of programs providing services in Snohomish County. Observations were 
related to program sites and community information. 

The review addressed five goals: 1) identify successful models of service; 2) learn about 
family preferences; 3) describe changes in children’s and family’s priorities and needs; 

4) guide and support new options in response to families’ priorities; and 5) develop ways 
to evaluate accomplishments and progress in the future. The final report, expected to be 
available in July 1 998, will include recommendations regarding future directions for 
Snohomish County birth to three services. 
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DATA INVENTORY 



A wealth of information is available in secondary data sources on children in the state of 
Washington. An inventory of relevant technical data was conducted to identify possible 
indicators for child and family early intervention outcomes, and to assess the feasibility of 
obtaining access to these data. 

Extensive information on children with special needs over the age of three is maintained in 
local school records. While school records are not readily accessible for research, both the 
Department of Health and Department of Social and Health Services (DSHS), Division of 
Developmental Disabilities maintain information for children to whom they provide 
services. Another source of information for children who are bom to Washington 
residents is contained in the First Steps Database. Additional databases include that 
maintained by the DSHS, Children’s Administration on child abuse, neglect and out-of- 
home placement. All of these databases can be linked to provide information on the 
background characteristics of children served, the type of services received, and some 
limited information relative to type of disability and outcome. Despite the plenitude of 
data, these secondary sources provide very limited information regarding child’s disability 
and the family environment. As a result of the limitations in these secondary data sources, 
all of the proposed research projects using these secondary sources involve additional 
primary data collection. 

DATA FROM OFFICE OF SUPERINTENDENT OF PUBLIC 
INSTRUCTION/STUDENT FILES 

The confidentiality issues surrounding access to student identified data are significant. 

The release of personally identifiable information (without parental consent) is permitted 
in limited cases: in order to meet the requirements imposed by chapter 392-172 WAC; or 
to.officials of participating agencies under the rules governing the Family Educational 
Rights and Privacy Act of 1974. In general, for written informed consent not to be 
required, the disclosure must be on behalf of school districts’ efforts to improve 
instruction and the manner in which the information will be used to improve instruction 
must be clearly articulated. Permission to obtain an enrollment count for three to six year 
olds must be negotiated with Office of Superintendent of Public Instruction (OSPI) and 
would require Human Research Review Board (HRRB) approval. Permission for access 
to children’s school records would have to be negotiated separately with each district or 
through written parental consent. With parental consent, HRRB approval, and support 
from OSPI it would be possible to review student files. Some of the information which 
would be contained in these student files is listed in Table 1. 
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DATA FROM DIVISION OF DEVELOPMENTAL DISABILITIES FILES 

The Division of Developmental Disabilities maintains information in two major databases, 
the Common Client Database (CCD) and Trends and Patterns Database (TPD). These 
databases contain information on child development services, eligibility for services and 
family support services including respite care, attendant care, and nursing care. 
Information is collected on where the services are delivered, where the child resides, and 
the type of residence. The eligibility file contains information on type of disability. While 
CCD does not retain that information historically, TPD contains snapshots, on that field 
indicating changes in the child’s eligibility over time. One caution in interpreting the 
service delivery data is that information reported by month tracks authorization and 
payment through SSPS rather than receipt of services. A day program file contains 
information on date placed and date where involvement in the day (child development) 
program ended. Some of the information in Division of Developmental Disabilities 
database files in listed in Table 2. 



DATA FROM DEPARTMENT OF HEALTH FILES 



The Department of Health maintains information from local health jurisdictions in the 
Child Health Intake Form (CHIF) Automated System. This system contains background 
information on the child and their diagnostic category, but it does not appear to track the 
number and type of services delivered or other variables which might be useful for an 
outcome based evaluation. Six neurodevelopmental centers are using the Wee Functional 
Indicator Measure (WeeFIM) to track outcomes, and additional centers may use it in the 
future. This is a potentially rich source of information. 
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FIRST STEPS DATABASE 

The First Steps Database contains information on all children bom to Washington 
residents starting in July 1988. It is updated on an annual basis and it takes about a year 
before the births from a given year are added to the database. It currently contains births 
through December 1 996. 



The First Steps Database was established to track maternity care and birth outcomes for 
low income women who received prenatal care through Medicaid and other state 
supported programs. It originally contained information from birth certificates, linked 
Medicaid claims for the prenatal period and the year postpartum, and information on infant 
deaths. Its purposes have expanded and now it is routinely linked to data systems 
developed by the Children’s Administration for tracking referrals for child abuse or neglect 
and for out-of-home placements and until 1996 it was annually linked to the substance 
abuse treatment encounter database (TARGET) maintained by the Division of Alcohol and 
Substance Abuse. In addition, information on child deaths through age six and enrollment 
in early intervention services have been linked to the First Steps Database for special 
projects. 



Selected information which is already in or linked to the First Steps Database for all 
children bom to Washington residents is listed in Table 3. For children whose mothers 
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had maternity care provided by the Medical Assistance Administration, additional 
information is available using these claims. The additional information available using 
claims is listed in Table 4. Other data elements which could be added to the First Steps 
Database by linking to other state data systems are listed in Table 5. 

Six counts of children enrolled in early intervention services for children under three years 
old have been conducted by Research and Data Analysis (RDA) and were funded by the 
Infant Toddler Early Intervention Program (ITEIP). Most of the analyses which RDA has 
been able to perform depend upon linking the enrollment data to the First Steps Database. 
Approximately 15 percent of children enrolled in early intervention services were not born 
to Washington residents and are not in the First Steps Database. However, RDA projects 
could link' those children to the data systems with information on child abuse and neglect, 
out-of-home placement and child and infant death. It is possible also to link those children 
to eligibility files in order to determine if they were covered by Medicaid. The limited 
information available on children not born to Washington residents is listed in Table 6. 



TABLE 1. DATA FROM SCHOOL DISTRICT STUDENT FILES 



Data 


Availability 


Notes | 


Enrollment in special 
education. Participation in 
inclusive settings 


Yes 


The files contain information on enrollment in 
special education, receipt of special services, 
and proportion of time in inclusive settings. 


Child functioning 


IFSP* 


Collected at the individual level by school 
districts for children birth to three with Family 
Resource Coordinators (FRCs). 


Severity of disability 


IEP** 


Individual evaluation for all children with 
disabilities in student files is conducted at the 
time of eligibility determination. 
Individualized Education Program contains 
information on proposed services, goals, and 
placement decisions. 



’"Individualized Family Service Plan 
’"’"Individualized Education Program 



TABLE 2. DATA FROM DIVISION OF DEVELOPMENTAL DISABILITIES 



Data 


Source 


Notes 


Transitioning out of services 


CCDB* 


The eligibility file contains information on 
whether or not a child still qualified for/needed 
services which can be combined with the 
information on disability type. Information on 
disability type is kept historically in the Trends 
and Patterns Database. 


Child residence 


CCDB* 


The residential file contains historical 
information on the type of residence of the 
child (institution, in-home, foster care etc.). 1 
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TABLE 3. BATA FOR CHILDREN BORN TO WASHINGTON RESIDENTS 
IN (OR ALREADY LINKED TO) THE FIRST STEPS DATABASE 



BACKGROUND 

VARIABLES 


SOURCE 


NOTES 


Mother’s marital status 


FSDB* 


from birth certificate — at time of birth 


Mother’s age 


FSDB* 


from birth certificate — at time of birth 


Mother’s education 


FSDB* 


from birth certificate — from 1992 on 


Eligibility for Medicaid 


FSDB* 


from OFM eligibility file — from conception 
through 2 years after 


Ethnicity 


FSDB* 


from birth certificate — as stated on birth 
certificate 


Receipt of prenatal pare 


FSDB* 


from birth certificate — month began prenatal 
care 


Parity 


FSDB* 


from birth certificate — number of prior 
pregnancies or prior children 


Amniocentesis 


FSDB* 


from birth certificate 


Birthweight 


FSDB* 


from birth certificate — grams 


Gestational age 


FSDB* 


from birth certificate — estimated gestational age 


Apgar score 


FSDB* 


from birth certificate 


Enrollment in the Infant Toddler 
Early Intervention Program 


RDA** 


enrollment in B-3 on Dec ’93, May ’95, Dec 
’95, May ’96, Dec ’96, Dec ‘97 


Condition: identified at birth 


FSDB* 


from birth certificate for conditions identified at 
birth 


j® 111 






DATA 


SOURCE 


NOTES 


Infant death 


FSDB* 


from matched infant death file for 7/88-12/95 


Child death 


FSDB* 


currently linked for child deaths prior to 1 996 
for children bom 7/91-12/94. 


Accepted referral to child and 
protective services 


CAMIS*** 


for children bom after July 1, 1991; CAMIS 
data linked to FSDB for referrals through 
August, 1996 to children bom before 1996 


Placement by children’s services 
out-of-home 


CAMIS*** 


for children bom after July 1, 1991; CAMIS 
data linked to FSDB for placements through 
August 1 996 to children bom before 1 996 



* First Steps Database 

** Research and Data Analysis 

***Case and Management Information System 
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TABLE 4. ADDITIONAL DATA FOR CHILDREN WITH MEDICAID 
ELIGIBILITY BORN TO WASHINGTON RESIDENTS, 7/1/1988-12/1/1996 



BACKGROUND VARIABLES 


SOURCE 


NOTES 


Maternal amniocentesis, CVS** 


FSDB* 


from Medicaid claims 


Maternal substance abuse 
diagnoses 


FSDB* 


from Medicaid claims for women with 
Medicaid eligibility 


Condition for Medicaid children 


FSDB* 


from Medicaid claims for children with 
Medicaid eligibility for the first 2 years of 
life 


Medicaid/SSI*** eligibility 


FSDB* 


from OFM eligibility file — for kids for 
first 2 years of life — could be obtained 
for more years 


Medicaid costs 


FSDB* 


from Medicaid claims — for non-Healthy 
Options kids 



"■First Steps Database 

** Chorionic Villa Sampling 

*** Supplemental Security Income 



TABLE 5. DATA WHICH COULD BE OBTAINED AND LINKED TO THE 
FIRST STEPS DATABASE FOR CHILDREN OF WASHINGTON RESIDENTS 



BACKGROUND VARIABLES 


SOURCE 


NOTES 


Employment data 


ESD* 


Quarterly wage data for mother and 
father (if listed on birth certificate) 


Child’s residence 


OFM** 


For children with Medicaid eligibility, 
most recent address could be obtained 
for last period of eligibility. 


Mother’s residence 


DOL*** 


Mother’s driver’s license information to 
confirm current Washington residency 
and local school district 


Medicaid diagnoses > 2 years 


MAA+ 


Medicaid claims could be obtained for 
children over two years old to flag 
children with later diagnosed conditions 


School district 


FSDB-GIS++ 


Birth certificate address used to identify 
school district. 



"■Employment Security Department 

* "‘Office of Financial Management 

** "‘Department of Licensing 

+Medical Assistance Administration 

-H-First Steps Database-Geographical Information System 
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| DATA 


SOURCE 


NOTES 


Divorce/Marriage 


DOH-CHS* 


Information on divorces and marriages 


I Medicaid/SSl** eligibility 


OFM*** 


For children who have any Medicaid 
eligibility, their continuance on Medicaid 
can be determined. 



♦Department of Health — Center for Health Statistics 
** Supplemental Security Income 
♦♦♦Office of Financial Management 



TABLE 6. DATA FOR ALL CHILDREN IN WASHINGTON STATE 
ENROLLED IN EARLY INTERVENTION SERVICES ON: 
12/1993, 5/1995, 12/1995, 5/1996, 12/1996, AND 12/1997 



Data 


Source 


Notes 


Accepted referral to child and 
protective services 


CAMIS* 


for children bom after July 1, 1991 


Placement by children’s services 
out-of-home 


CAMIS* 


for children bom after July 1, 1991 


Infant or child death 


DOH-CHS** 


currently linked for child deaths prior to 
1996 for children bom 7/91-12/94 


Medicaid/SSI+ eligibility 


OFM*** 


Children can be linked to the OFM files 
to determine Medicaid eligibility, 
including children not bom to 
Washington residents. 



♦Case and Management Information System 
** Department of Health — Center for Health Statistics 
♦♦♦Office of Financial Management 
+Supplemental Security Income 
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CHAPTER 7 



STUDY DESIGNS 

This chapter presents possible options for study designs for evaluating the impact of early 
intervention on infants and toddlers, birth to three, with disabling or delaying conditions 
and their families in Washington State. Following a brief literature review, each study is 
described in full. Each study design includes the following: a description of the 
methodology, suggestions for child and family outcomes, suggested analysis techniques, a 
discussion of strengths and weaknesses, and estimated duration and costs of the study. 

GENERAL LITERATURE REVIEW 

The literature regarding the effectiveness of early intervention for children with disabling 
or delaying conditions and their families should be reviewed within the context of the field 
of child development itself. One impetus behind early intervention legislation was the 
emergence of data and developmental theories that indicated the unique intervention 
potential during the first few years of life (Futterweit and Ruff, 1993). As was previously 
discussed, development of neural synapses which form patterns of behavior and learning 
style occurs as early as infancy (Huttenlocher, 1990). Three populations of children are the 
targets of early intervention services: 1) those with established developmental delays or 
disabilities; 2) those at biomedical risk (such as low birth weight), and 3) those at 
environmental risk (such as those existing in poverty) (Tjossem, T976). One of the aims 
of early intervention is to maximize each child’s developmental potential. While it is 
generally accepted that early intervention is effective in stimulating development (Casto 
and Mastropieri, 1986; Guralnick, 1998), understanding the relationship between child and 
family factors, program factors, and types of outcomes desired is still being researched 
(Guralnick, 1998). 

The questions of early intervention research — for whom does early intervention work, 
under what conditions, and toward what goals (Guralnick, 1988) — are shaped within the 
evolution of the field of child development itself. Early studies assessing intervention 
effects defined development as a strictly cognitive construct, more specifically, 
psychometrically defined intelligence (Guralnick, 1998). Current studies reflect trends in 
the field of child development which suggest that development should be construed in a 
much broader sense to include an array of social factors (Guralnick, 1997; Roberts et al., 
1997). Similarly, earlier studies focused on child development as the sole outcome of 
intervention (Innocenti, 1996), while contemporary thought suggests that development is 
an interactive activity and emphasizes the importance of the family and social environment 
(Boyce, 1993; Casto and White, 1993; Crowley and Taylor, 1993:4; Gallagher, 1996). 
With the growth in emphasis on family focused early intervention strategies, recent studies 
have incorporated family outcomes as an integral part of intervention assessment (Bailey 
et al., 1998; Caro and Derevensky, 1991; Innocenti, 1996). As concepts of development 
and strategies of intervention evolve, the parameters of research change. It is within this 
context that a brief, general review of the literature will be presented. The individual 
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discussion of study designs contains a literature review section which elaborates on studies 
of that specific nature. 

First Generation Research 

Early intervention research is often categorized into two time periods. First generation 
research indicates research occurring from the early 60s until the late 80s; second 
generation research indicates research from the late 80s until the present. 

The focus of first generation research has primarily addressed the issue of the general 
effectiveness of early intervention services. Many first generation studies have used at risk 
or economically disadvantaged populations to draw conclusions about the impact of early 
intervention programs (Casto and Mastropieri, 1986). These studies produced strong 
evidence that early intervention programs had a positive impact (Innocenti, 1996). Their 
results were supported by longitudinal studies which confirmed early intervention 
effectiveness in the long term such as the following: the Consortium for Longitudinal 
Studies (Lazar and Darington, 1982), the Perry Preschool Project (Berrueta-Clement et 
al., 1984), and the Abecedarian Project (Ramey and Haskins, 1981). Some researchers 
have questioned whether the results from studies using at risk, preschool aged children can 
be applied to those children, birth to three, with established disabling or delaying 
conditions (White and Casto, 1985). Despite these concerns, studies that did focus on 
birth to three populations of disabling or delaying conditions generally showed positive 
results (Guralnick and Bennet, 1987; Lovaas, 1987; McEachlin et al., 1993). Below are 
listed results from some reviews of first generation research pertaining to those with 
disabling or delaying conditions. 

• In a review of 42 first generation projects researching the impact of early intervention 
services on the disabled population, Farran (1990) stated that there were short and 
long term gains in all areas of child development. 

• In a review of 74 studies reviewing the impact of early intervention on children with 
disabling conditions, Casto and Mastopieri (1986) reported that participants in early 
intervention services experienced improvement in IQ, motor, and language skills. 

• In a review of 21 demonstration studies for infants and children with Down Syndrome, 
Gibson and Harris (1988) noted that early intervention services had a positive impact 
on fine motor skills, IQs, and social behavior. 

• In a review of 1 1 studies researching the impact of early infant stimulation programs 
on infants and children with Down Syndrome, Gibson and Fields (1984) reported gains 
in developmental skills and social habits. 

• In a review of 74 studies for children with disabling conditions, Casto (1986) stated 
that “early intervention programs result in moderately large immediate benefits” in 
outcomes such as cognitive, language, motor, and social skills, and parent-child 
interaction. 





® White, et al. (1985) conducted a review of the reviewers of first generation (until 
1985) studies and reported that 94% of 52 primary literature reviewers concluded that 
early intervention services resulted in “substantial immediate benefits for handicapped, 
at risk, and disadvantaged populations.” 

First generation research faced a host of methodological problems which include the 
following: 1) lack of random assignment, 2) subject selection, and 3) problems of 
measurement (Bricker et al., 1984; Farran, 1990; Guralnick, 1991). Second generation 
research faces similar difficulties. These problems will be discussed in the subsequent 
section. Methodological Challenges. While these methodological problems have affected 
a large number of studies (Casto and Mastropieri, 1986; ShonkofF and Hauser-Cram, 

1 987), it is generally accepted that early intervention has a positive impact for children 
with disabling or delaying conditions (Casto and Mastropieri, 1 986; Guralnick 1 997; 
ShonkofF and Hauser-Cram, 1 987), and that first generation research has “put the global 
question about whether or not early intervention is effective to rest” (Guralnick, 1997). 



Second Generation Research 

Second generation research attempts to respond to the shortcomings of first generation 
research and incorporates current developments in the field of early childhood 
development. First generation research measured the effectiveness of early intervention 
programs from the early 60s to the late 80s. Programs at this time were newly developed, 
uncoordinated and fragmented (Guralnick, 1997). As a result, which element of early 
intervention was effective (or not) was not often discernible in earlier studies. Earlier 
studies also failed to take into account established independent variables that may affect 
the developmental progress of a child and family. Without taking these into account, 
conclusions regarding the impact of early intervention may be erroneous (ShonkofF et al., 
1988). As a result, second generation research has focused on ascertaining which 
elements of early intervention have a positive impact on which type of subjects under what 
circumstances. A typical study of this type is Longitudinal Studies of the Effects of 
Alternative Types of Early Intervention for Children with Disabilities (Innocenti, 1996). 



While first generation research focused primarily on cognitive child development, the 
concept of development has been widened to include skills that will allow children to 
participate in valued roles in the community, perform to the best of their ability in school 
and be able to become contributing members of society. T o assess these goals, outcomes 
other than IQ, such as adaptive, communication, and social skills have become prevalent 
(Casto and Lewis, 1984; Guralnick, 1991 and 1989; Hauser-Cram and ShonkofF, 1988). 




The growing emphasis on the importance of the effect of family and social environment on 
a child’s development has expanded the concept of development even further. While 
previous studies focused solely on child outcomes, current research has begun to discuss 
and assess impact of early intervention on families especially in the areas of family support, 
parent-child interaction, family empowerment, and knowledge needed to enhance a child’s 
potential for development (Boyce, 1993; Casto and White, 1993; Cleary, 1988; Crowley 
and Taylor, 1993:4; Gallagher, 1996, Kelly and Barnard, in press). 
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In sum, second generation research has moved beyond the general impact question and 
attempts to identify the “child characteristics, family characteristics, and program features 
that interact to optimize one or more outcomes within the framework of contemporary 
early intervention services” (Guralnick, 1993). 

Some General Results of Second Generation Research 



Some general results of second generation research are listed below. These conclusions 
were drawn by the referenced authors after a review of second generation studies. The 
projects were randomized controlled studies or met the criterion of “believability” which 
McLean and Cripe (1997) describe as requiring that a “skeptical reviewer would believe 
that the study’s results were almost certainly attributable to the experimental intervention, 
and not to maturation or some other uncontrolled variable” (McLean and Cripe, 1997). 



• Early intervention is effective in training mothers to be more responsive to infants with 
developmental delays (Spiker and Hopmann, 1997). 

• Infant stimulation had a positive impact on motor skills for children with cerebral palsy 
or related motor delays, especially when parent mediated (Harris, 1997). 

• Therapeutic treatment for young children (birth to four years) with cerebral palsy or 
motor delays has a greater impact on development than for older (four to eight years) 
children with cerebral palsy or related motor delays (Harris, 1997). 

• Parent understanding of and compliance with therapeutic intervention programs 
positively affect the development of fine motor skills of children with cerebral palsy 
and motor disabilities (Harris, 1997). 

• Early intervention programs using the Mother-Infant Communication Program 1 
positively affect maternal responsivity to children with communication disorders 
(McLean and Cripe, 1997). 

• Families using Hanen Early Language Parent Program 2 increased the frequency and 
duration of joint and interactive engagement with their children diagnosed with 
communication delays than families who did not use this model (McLean and Cripe, 
1997). 

• More successful treatment with greater efficiency has occurred when children with 
communication disorders are enrolled in early intervention at a young age (McLean 
and Cripe, 1997). 

• Parents can serve as effective interventionist for children with communication 
disorders (McLean and Cripe, 1997). 



1 This program emphasizes strengths of mothers by focusing on behaviors already being used in order to 
increase maternal responsivity to children with communication delays. It incorporates nondirective 
strategies such as modeling, video feedback, and planned practice (McLean and Cripe, 1997). 
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